The present invention relates generally to interface devices between humans and computers, and more particularly to computer interface devices that provide force feedback to the user.
Computer systems are used extensively in many different industries to implement computer controlled simulations, games, and other application programs. More particularly, these types of games and simulations are very popular with the mass market of home consumers. A computer system typically displays a visual environment to a user on a display screen or other visual output device. Users can interact with the displayed environment to play a game, experience a simulation or xe2x80x9cvirtual realityxe2x80x9d environment, or otherwise influence events or images depicted on the screen. Such user interaction can be implemented through the use of a human-computer interface device, such as a joystick, xe2x80x9cjoypadxe2x80x9d button controller, mouse, trackball, stylus and tablet, or the like, that is connected to the computer system controlling the displayed environment. The computer updates the simulation or game in response to the user""s manipulation of an object such as a joystick handle or mouse, and provides feedback to the user utilizing the display screen and, typically, audio speakers.
In some interface devices, tactile (xe2x80x9chapticxe2x80x9d) feedback is also provided to the user, more generally known as xe2x80x9cforce feedback.xe2x80x9d These types of interface devices can provide physical sensations to the user manipulating the object of the interface device. Typically, motors or other actuators are coupled to the object and are connected to the controlling computer system. The computer system can provide forces on the object in conjunction with simulation/game events by sending control signals to the actuators. The computer system can thus convey physical sensations to the user in conjunction with other supplied feedback as the user is grasping or contacting the object of the interface device. Force feedback interface devices can thus provide a whole new modality for human-computer interaction.
Force feedback input/output (I/O) devices of the prior art have concentrated on providing maximum haptic fidelity, i.e., the realism of the tactile feedback was desired to be optimized. This is because most of the force feedback devices have been targeted at the specific needs of highly industrial applications, and not a mass consumer market. To attain such realism, mass market design concerns such as low size and weight, low complexity, programming compatibility, low cost, and safety have been sacrificed in the prior art. As a result, typical force feedback interface devices include complex robotic mechanisms which require precision components and expensive actuators.
An important concern for a force feedback interface device is communication bandwidth between the controlling computer and the interface device. To provide realistic force feedback, the complex devices of the prior art typically use high speed communication electronics that allow the controlling computer to quickly update force feedback signals to the interface device. The more quickly the controlling computer can send and receive signals to and from the interface device, the more accurately and realistically the desired forces can be applied on the interface object. In addition, using a high bandwidth communication interface, force feedback can be accurately coordinated with other supplied feedback, such as images on the video screen, and with user inputs such as movement of the object, activated buttons, etc. For example, a user can grasp and move a force feedback joystick in a simulation to control an image of a car to drive over a virtual bumpy surface displayed on a screen. The controlling computer should provide control signals to the actuators of the joystick quickly enough so that the surface feels as realistically bumpy as the designer of the simulation intended. If the control signals are too slow, a realistic feeling of bumpiness is more difficult to provide. Also, the controlling computer needs a high bandwidth communication interface to accurately coordinate the supplied forces with the visual feedback on the screen, such as the moment on the screen when the car first contacts the bumpy surface. This high speed is likewise needed to accurately coordinate supplied forces with any input from the user, for example, to steer the car in particular directions.
A problem is evident when prior art force feedback interface devices are provided to the mass consumer market. Most home computers have a built-in standard serial communication interfaces, such as an RS-232 or RS422 interface, that may conveniently be used to connect peripherals like a force feedback interface device to the host computer. In addition, manufacturers prefer to provide peripheral devices that use these serial interfaces, since no additional hardware, such as interface cards, needs to be provided with such peripherals. The manufacturing cost of the peripheral device can thus be significantly reduced. However, these standard serial communication interfaces are typically quite slow (i.e. have low bandwidth) compared to other communicaton interfaces. Realistic and accurate force feedback thus becomes difficult to provide by a controlling computer system to a prior art interface device connected through such a serial interface. For example, U.S. Pat. No. 5,184,319, by J. Kramer, describes a force feedback device that applies forces to a user""s body parts. However, the Kramer device is typical of the prior art in that the host computer directly controls the actuators and directly receives the sensor data from the interface apparatus. Such a device is not suitable for a low bandwidth communication interface to achieve realistic force feedback.
Another problem with using prior art force feedback interface devices in the mass consumer market is the wide variety of computer platforms and processing speeds that are used on different computers and on the same computer at different times. The force sensations provided to a user by a force feedback interface device may feel different to a user on different computer platforms or microprocessors, since these different computers run at different speeds. For example, the force feedback controlled by a 100 MHz computer may be much different from the force feedback controlled by a 60 MHz computer due to the different rates of processing control signals, even though these forces are intended to feel the same. In addition, the effective processing speed of one microprocessor can vary over time to provide inconsistent forces over multiple user sessions. For example, multitasking can vary or delay a microprocessor""s management of force feedback control signals depending on other programs that are running on the microprocessor.
In addition, there is no standardized language or communication protocol for communicating with force feedback devices. A software developer that wishes to provide force feedback to an interface in a software application must currently set up his or her own specialized commands and/or communications protocols and must implement the force feedback controlling instructions at a low level. This requires unnecessary time and expense in the development of software applications that include features directed toward force feedback interfaces.
Therefore, a more realistic, cost effective, and standardized alternative to force feedback interfaces and force feedback control paradigms is desired for certain applications.
The present invention is directed to controlling and providing force feedback to a user operating a human/computer interface device. The interface device is connected to a controlling host computer and includes a separate microprocessor local to the interface device. The local microprocessor receives high-level host commands and implements independent reflex processes.
More particularly, a system and method of the present invention for controlling an interface apparatus manipulated by a user includes a host computer system for receiving an input control signal and for providing a host command. The host computer updates a host application process, such as a simulation or video game, in response to the input control signal. A microprocessor local to the interface apparatus and separate from the host computer receives the host command and provides a processor output control signal. An actuator receives the processor output control signal and provides a force along a degree of freedom to a user-manipulated object, such as a joystick, in accordance with the processor output control signal. A sensor detects motion of the user object along the degree of freedom and outputs the input control signal including information representative of the position and motion of the object. Preferably, the sensor outputs the input control signal to the local microprocessor, which outputs the input control signal to the host computer. The user object is preferably grasped and moved by the user, and can include such articles as a joystick, mouse, simulated medical instrument, stylus, or other object. The user object can preferably be moved in one or more degrees of freedom using, for example, a gimbal or slotted yoke mechanism, where an actuator and sensor can be provided for each degree of freedom.
The application process updated by the host computer system preferably includes application software that can be simulation software, game software, scientific software, etc. The host computer system displays images on a visual output device such as a display screen and synchronizes the images and visual events with the position and motion of the user object as well as forces applied to the object. The present invention can use a standard serial interface included on many computers to interface the host computer system with the local microprocessor. A clock is preferably coupled to the host computer system and/or the local processor which can be accessed for timing data to help determine the force output by the actuator.
In the preferred xe2x80x9creflexxe2x80x9d embodiment, the host computer receives the sensor information in a supervisory mode and outputs a high level host command to the microprocessor whenever a force is required to be applied to the user object or changed. In accordance with the high level host command, the microprocessor reads sensor and timing data and outputs force values to the actuator according to a reflex process that is selected. The reflex process can include using force equations, reading force profiles of predetermined force values from a storage device, or other steps, and may be dependent on sensor data, timing data, host command data, or other data. The processor thus implements a reflex to control forces independently of the host computer until the host computer changes the type of force applied to the user object.
The invention also provides a paradigm for force commands between the host computer and the local microprocessor. The high level host commands provided by the host computer can be rate control and/or position control commands, and may include information in the form of command parameters. By providing a relatively small set of commands and command parameters which are translated into a panoply of forces, the paradigm further shifts the computational burden from the host computer to the local microprocessor. Host commands may include commands to provide forces on the user object such as restoring forces, vibration forces, texture forces, a barrier forces, attractive/repulsive force fields, damping forces, groove forces, and a paddle-ball force. Typical command parameters include a magnitude parameter, a duration parameter, a direction parameter, a style parameter, and a button parameter to control the force output by the actuator. This provides a high level, standard force feedback command protocol for the efficient use by developers of force feedback software to be implemented on the host computer system.
A preferred implementation of the functionality of the local microprocessor is also provided. A command process receives a host command from the host computer and processes the host command and any command parameters included in the host command. Force parameters are derived from the host command and the command parameter and are stored in memory. Preferably, every host command has a set of force parameters associated with it to be updated when the appropriate host command is received. A status update process reads sensor data from the sensors describing a motion of the user object. The status update process can also compute velocity, acceleration, or other time-related values if appropriate. A force output process computes a force value using a reflex process selected in accordance with the force parameters and the sensor data. In some instances, the force value may depend on the values of the force parameters and sensor data. The force output process outputs a force on the user object by sending the computed force value to the actuators. In addition, a reporting process reports the sensor data to the host computer system when appropriate. Preferably, a plurality of host commands can be in effect simultaneously, where a force value is summed from a reflex process corresponding to each such host command in effect. Also, parameter page(s) of sets of parameters can conveniently be stored in memory to allow different force environments to be selected.
The control system and method of the present invention advantageously includes a separate processor local to the interface device that is separate from the host computer system. The local processor can read and process sensor signals as well as output force command signals independently of the host computer, thus saving significant processing time on the host computer. In addition, the use of the local processor to handle low-level force feedback commands allows more realistic and accurate force feedback to be provided to an object manipulated by a user when using a serial or other relatively low-bandwidth communication interface between the host and the interface device, since such low level force signals do not have to be transmitted over the communication interface. The use of specific, high level host commands to command a variety of types of forces allows force feedback host applications to be easily created. These improvements allow a computer system to provide accurate and realistic force feedback over a low-cost, low bandwidth communication interface and is thus ideal for the mass market of home computer systems.
These and other advantages of the present invention will become apparent to those skilled in the art upon a reading of the following specification of the invention and a study of the several figures of the drawing.